Processing method and processing system for multiple depth information

ABSTRACT

A processing method and a processing system for multiple depth information are provided. The processing method for multiple depth information includes the following steps. A plurality of first images and a plurality of second images are obtained. The first images and the second images are inputted to the same depth generating unit. The first images and the second images are calculated by the depth generating unit to obtain a plurality of depth information corresponding to the first images and the second images.

This application claims the benefit of U.S. provisional application Ser.No. 63/011,246, filed Apr. 16, 2020, the subject matter of which isincorporated herein by reference.

BACKGROUND OF THE INVENTION Field of the Invention

The invention relates in general to a processing method and a processingsystem and more particularly to a processing method and a processingsystem for multiple depth information.

Description of the Related Art

Along with the rapid advance in display technology, a 3D displaytechnology has been provided. According to the 3D display technology, aleft-eye original image and a right-eye original image can be inputtedto a depth engine to calculate depth information. When severalresolutions or baselines are required, several depth engines are neededto calculate several sets of left-eye and right-eye original images togenerate multiple depth information.

SUMMARY OF THE INVENTION

The present disclosure relates to a processing method for multiple depthinformation. The processing method for multiple depth informationincludes the following steps. A plurality of original images areobtained. A plurality of first images and a plurality of second imagesare obtained according to the original images. The first images and thesecond images are inputted to the same depth generating unit. The firstimages and the second images are calculated by the depth generating unitto obtain a plurality of depth information. A processing method and aprocessing system for multiple depth information are provided. Theprocessing method for multiple depth information includes the followingsteps. A plurality of first images and a plurality of second images areobtained. The first images and the second images are inputted to thesame depth generating unit. The first images and the second images arecalculated by the depth generating unit to obtain a plurality of depthinformation corresponding to the first images and the second images.

According to another embodiment of the present disclosure, a processingsystem for multiple depth information is provided. The processing systemincludes an image capturing module and a depth generating unit. Theimage capturing module is used to obtain a plurality of first images anda plurality of second images. The depth generating unit is used toreceive and calculate the first images and the second images to obtain aplurality of depth information corresponding to the first images and thesecond images.

The above and other aspects of the invention will become betterunderstood with regard to the following detailed description of thepreferred but non-limiting embodiment(s). The following description ismade with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of a processing system for multipledepth information according to an embodiment.

FIG. 2 illustrates a flowchart of a processing method for multiple depthinformation according to an embodiment.

FIG. 3A illustrates a schematic diagram of image capturing unitsaccording to an embodiment.

FIG. 3B illustrates a conversion of original images.

FIG. 4A illustrates a combination of the first images and the secondimages according to an embodiment.

FIG. 4B illustrates a combination of the first images and the secondimages according to another embodiment.

FIG. 4C illustrates a combination of the first images and the secondimages according to another embodiment.

FIG. 4D illustrates a combination of the first images and the secondimages according to another embodiment.

FIG. 4E illustrates a combination of the first images and the secondimages according to another embodiment.

FIG. 4F illustrates a combination of the first images and the secondimages according to another embodiment.

FIG. 4G illustrates a combination of the first images and the secondimages according to another embodiment.

FIG. 5A illustrates a calculation result obtained from the input of thecomposite image of FIG. 4A.

FIG. 5B illustrates a calculation result obtained from the input of thecomposite image of FIG. 4B.

FIG. 5C illustrates a calculation result obtained from the input of thecomposite image of FIG. 4C.

FIG. 5D illustrates a calculation result obtained from the input of thecomposite image of FIG. 4D.

FIG. 5E illustrates a calculation result obtained from the input of thecomposite image of FIG. 4E.

FIG. 5F illustrates a calculation result obtained from the input of thecomposite image of FIG. 4F.

FIG. 5G illustrates a calculation result obtained from the input of thecomposite image of FIG. 4G.

DETAILED DESCRIPTION OF THE INVENTION

Referring to FIG. 1, a block diagram of a processing system 100 formultiple depth information according to an embodiment is shown. Theprocessing system 100 includes an image capturing module (notillustrated), a depth generating unit 130 and a decomposition unit 140.In the embodiment of the invention, the image capturing module includesa plurality of image capturing units 110, a conversion unit 111 and acombination unit 120. The image capturing units 110 are used to captureand process color images, grayscale images or infra-red images. Theimage capturing units 110 can be realized by such as color cameras,black and white cameras or infra-red cameras. The quantity of the imagecapturing units 110 can be two or above. The conversion unit 111, thecombination unit 120, the depth generating unit 130 and/or thedecomposition unit 140 can be realized by such as programming codes,circuits, chips, circuit boards or storage devices for storingprogramming codes.

The image capturing units 110 simultaneously obtain images IA, IB andIC. The same depth generating unit 130 can calculate the images IA, IBand IC to obtain depth information D1 and D2. Details of the processingmethod for multiple depth information of the present embodiment aredisclosed below with an accompanying flowchart.

Referring to FIG. 2, a flowchart of a processing method for multipledepth information according to an embodiment is shown. Firstly, themethod begins at step S100, original images IA, IB and IC are obtainedby the image capturing units 110. Referring to FIG. 3A, a schematicdiagram of image capturing units 110 according to an embodiment isshown. In the example of FIG. 3A, the quantity of the image capturingunits 110 is exemplified by 3. The image capturing unit 110 designatedby “B” is disposed between the image capturing unit 110 designated by“A” and the image capturing unit 110 designated by “C”. The baseline Babis located between the image capturing unit 110 designated by “A” andthe image capturing unit 110 designated by “B”. The baseline Bac islocated between the image capturing unit 110 designated by “A” and theimage capturing unit 110 designated by “C”. The baseline Bac is greaterthan the baseline Bab. The image capturing unit 110 designated by “A” isused to obtain the original image IA. The image capturing unit 110designated by “B” is used to obtain the original image IB. The imagecapturing unit 110 designated by “C” is used to obtain the originalimage IC. The image capturing units 110 simultaneously obtain theoriginal images IA, IB and IC.

Then, the method proceeds to step S110, first image IAL and secondimages IBR and ICR are obtained by the conversion unit 111 according tothe original images IA, IB and IC. Referring to FIG. 3B, a conversion oforiginal images IA, IB and IC is shown. The first image IAL and thesecond images IBR and ICR can be obtained from the original images IA,IB and IC by a de-warping process, a rectification process and aresolution adjustment process. The first image IAL can be a left-eyeimage; the second images IBR and ICR can be right-eye images.

Then, the method proceeds to step S120, the first image IAL and thesecond images IBR and ICR are combined by the combination unit 120 toobtain a composite image C0. Referring to FIG. 4A, a combination of thefirst image IAL and the second images IBR and ICR according to anembodiment is shown. In the embodiment of FIG. 4A, two first images IALare arranged in a vertical direction and the second images IBR and ICRare arranged in a vertical direction to obtain a composite image C0.That is, the first image IAL and the second image IBR are arranged in ahorizontal direction; the first image IAL and the second image ICR arearranged in a horizontal direction. Such arrangement is adopted by thedepth generating unit 130 during the horizontal scanning calculation.

In the present step, the images IA, IB and IC can have the sameresolution and can be converted into the first image IAL and the secondimages IBR and ICR without adjusting the resolution, such that the firstimage IAL and the second image IBR maintain the same resolution, and thefirst image IAL and the second image ICR also maintain the sameresolution.

Referring to FIG. 4B, a combination of the first image IAL and thesecond images IBR and ICR according to another embodiment is shown. Inthe embodiment of FIG. 4B, two first images IAL are arranged in ahorizontal direction, and the second images IBR and ICR are arranged ina horizontal direction to obtain a composite image C0. That is, thefirst image IAL and the second image IBR are arranged in a verticaldirection, and the first image IAL and the second image ICR are alsoarranged in a vertical direction. Such arrangement is one of thepossible implementations.

Referring to FIG. 4C, a combination of first images IAL1 and IAL2 andsecond images IBR and ICR according to another embodiment is shown. Inanother embodiment, the first images IAL1 and IAL2 can be two differentfirst images obtained from the same original image IA adjusted using twodifferent adjustment parameters. Or, the first image IAL1 and the firstimage IAL2 can be two different first images obtained from the sameoriginal image IA processed by two different processing procedures (suchas different colors, brightness levels or formats).

Referring to FIG. 4D, a combination of the first images IAL′ and IAL,the second image IBR′, the second image ICR according to anotherembodiment is shown. In the embodiment of FIG. 4D, the first images IAL′and IAL have different resolutions (the resolution of the first imageIAL′ is lower than that of the first image IAL), and the second imageIBR′ and the second image ICR also have different resolutions (theresolution of the second image IBR′ is lower than that of the secondimage ICR). In FIG. 4D, the first images IAL′ and IAL can be twodifferent first images obtained from the same original image IA bydifferent resolution adjustment procedures. Or, the first image IAL′ canbe obtained from the original image IA by a resolution adjustmentprocedure, and the first image IAL and the original image IA maintainthe same resolution. In FIG. 4D, the slash shade represents null dataDK. The combination unit 120 combines the first images IAL′ and IAL, thesecond image IBR′, the second image ICR and the null data DK to obtain acomposite image C0.

Referring to FIG. 4E, a combination of the first images IAL′ and IAL,the second image IBR′, the second image ICR according to anotherembodiment is shown. In the embodiment of FIG. 4E, the first image IAL′is interposed in the null data DK, and the second image IBR′ is alsointerposed in the null data DK. The positions of the first image IAL′and the second image IBR′ are not restricted as long as the combinationunit 120 can provide a flag (such as a parameter or a signal) capable ofinforming subsequent elements of the positions of the first image IAL′and the second image IBR′.

Referring to FIG. 4F, a combination of the first images IAL and IAL′,the second images IBR and IBR′ according to another embodiment is shown.In the embodiment of FIG. 4F, the first image IAL and the first imageIAL′ have different resolutions (the first image IAL′ the resolution ofis lower than that of the first image IAL), and the second image IBR′and the second image ICR also have different resolutions (the resolutionof the second image IBR′ is lower than that of the second image ICR). InFIG. 4F, the first images IAL′ and IAL can be two different first imagesobtained from the same original image IA by different resolutionadjustment procedures. Or, the first image IAL′ can be obtained from theoriginal image IA by a resolution adjustment procedure. The resolutionof the first image IAL maintains the same with that of the originalimage IA. Similarly, the second image IBR′ and the second image IBR canbe two different second images obtained from the same original image IBby different resolution adjustment procedures. Or, the second image IBR′can be obtained from the original image IB by a resolution adjustmentprocedure, and the second image IBR and the original image IB maintainthe same resolution. In the present step, the first images IAL and IAL′and the second images IBR and IBR′ can be obtained from two originalimages IA and IB′ generated by two image capturing units 110.

Referring to FIG. 4G, a combination of the first image IAL and thesecond images IBR and ICR according to another embodiment is shown. InFIG. 4G, the first image IAL, the second image IBR, the first image IAL,the second image ICR are sequentially arranged in a horizontaldirection. Such arrangement is adopted by the depth generating unit 130during the horizontal scanning calculation.

Then, the method proceeds to step S130, the composite image C0 isinputted to the same depth generating unit 130.

Then, the method proceeds to step S140, the composite image C0 iscalculated by the depth generating unit 130 to obtain a calculationresult D0.

Referring to FIG. 5A, a calculation result D0 obtained from the input ofthe composite image C0 of FIG. 4A is shown. In the composite image C0 ofFIG. 4A, two first images IAL are arranged in a vertical direction, andthe second images IBR and ICR are arranged in a vertical direction,therefore in the calculation result D0 outputted from the depthgenerating unit 130, the depth information D1 and D2 will also bearranged in a vertical direction.

Referring to FIG. 5B, a calculation result D0 obtained from the input ofthe composite image C0 of FIG. 4B is shown. In the composite image C0 ofFIG. 4B, two first images IAL are arranged in a horizontal direction,and the second images IBR and ICR are arranged in a horizontaldirection, therefore in the calculation result D0 outputted from thedepth generating unit 130, the depth information D1 and D2 will also bearranged in a horizontal direction.

Referring to FIG. 5C, a calculation result D0 obtained from the input ofthe composite image C0 of FIG. 4C is shown. In the composite image C0 ofFIG. 4C, the first image IAL1 and the first image IAL2 are arranged in avertical direction, and the second images IBR and ICR are also arrangedin a vertical direction, therefore in the calculation result D0outputted from the depth generating unit 130, the depth information D1and D2 will also be arranged in a vertical direction.

Referring to FIG. 5D, a calculation result D0 obtained from the input ofthe composite image C0 of FIG. 4D is shown. In the composite image C0 ofFIG. 4D, the resolution of the first image IAL′ is lower than that ofthe first image IAL, and the resolution of the second image IBR′ islower than that of the second image ICR, therefore in the calculationresult D0 outputted from the depth generating unit 130, the resolutionof the depth information D1 will also be lower than that of depthinformation D2.

Referring to FIG. 5E, a calculation result D0 obtained from the input ofthe composite image C0 of FIG. 4E is shown. In the composite image C0 ofFIG. 4E, the first image IAL′ is interposed in the null data DK, and thesecond image IBR′ is also interposed in the null data DK, therefore inthe calculation result D0 outputted from the depth generating unit 130,the depth information D1 will also be interposed in the null data DK.

Referring to FIG. 5F, a calculation result D0 obtained from the input ofthe composite image C0 of FIG. 4F is shown. In the composite image C0 ofFIG. 4F, the resolution of the first image IAL′ is lower than that ofthe first image IAL, and the resolution of the second image IBR′ islower than that of the second image ICR, therefore in the calculationresult D0 outputted from the depth generating unit 130, the resolutionof depth information D2 will also be lower than that of the depthinformation D1.

Referring to FIG. 5G, a calculation result D0 obtained from the input ofthe composite image C0 of FIG. 4G is shown. In the composite image C0 ofFIG. 4G, the first image IAL, the second image IBR, the first image IALand the second image ICR are sequentially arranged in a horizontaldirection, therefore in the calculation result D0 outputted from thedepth generating unit 130, the depth information D1 and D2 will also bearranged in a horizontal direction.

Then, the method proceeds to step S150, the calculation result D0 isdecomposed by the decomposition unit 140 to obtain the depth informationD1 and D2. The decomposition unit 140 can obtain the positions and scopeof the the depth information D1 and D2 according to a flag (such as aparameter or a signal) to decompose the depth information D1 and D2. Or,the decomposition unit 140 can decompose the depth information D1 and D2according to the boundaries recognized by using the image processingtechnology.

Besides, the depth generating unit 130 can also perform calculation in atime-division multiplexing manner without using the combination unit 120or the decomposition unit 140. For example, when the first image IAL andthe second images IBR and ICR are inputted to the depth generating unit130, a flag (such as a parameter or a synchronous signal) can beinserted to the first image IAL and the second images IBR and ICR, suchthat the depth generating unit 130 can regard the first image IAL andthe second images IBR and ICR as two separate images and can directlyprocess the two separate images to output the two pieces of depthinformation D1 and D2 without performing combination or decomposition.

No matter time-division multiplexing is adopted or not, the originalimages IA, IB and IC are simultaneously obtained. The depth informationD1 and D2, which are lastly obtained, still correspond to the same timepoint. The images have very low time gap therebetween and are suitablefor fast moving objects.

In the embodiment, the image capturing module includes a plurality ofimage capturing units 110, a conversion unit 111 and a combination unit120. However, in other embodiments of the present invention, theconversion units and the combination unit of the image capturing modulecan be combined. In another embodiment of the present invention, theimage capturing units, the conversion unit and the combination unit ofthe image capturing module can also be combined.

According to the embodiment, multiple depth information can be generatedby the same depth generating unit 130 without incurring extra cost. Thedesign of using the same depth generating unit 130 makes hardware designmore flexible.

While the invention has been described by way of example and in terms ofthe preferred embodiment(s), it is to be understood that the inventionis not limited thereto. On the contrary, it is intended to cover variousmodifications and similar arrangements and procedures and the scope ofthe appended claims therefore should be accorded the broadestinterpretation so as to encompass all such modifications and similararrangements and procedures.

What is claimed is:
 1. A processing method for multiple depthinformation, comprising: obtaining a plurality of first images and aplurality of second images; inputting the first images and the secondimages to a depth generating unit; and calculating the first images andthe second images by the depth generating unit to obtain a plurality ofdepth information corresponding to the first images and the secondimages.
 2. The processing method for multiple depth informationaccording to claim 1, further comprising the following steps:simultaneously capturing a plurality of original images; and obtainingthe first images and the second images according to the original images.3. The processing method for multiple depth information according toclaim 1, wherein a quantity of the first images is two, a quantity ofthe second images is two, and the original images are captured by two orthree image capturing units.
 4. The processing method for multiple depthinformation according to claim 1, wherein the first images havedifferent resolutions and the second images have different resolutions.5. The processing method for multiple depth information according toclaim 1, further comprising: combining the first images and the secondimages to obtain a composite image; wherein the depth generating unitobtains a plurality of depth information corresponding to the firstimages and the second images according to the composite image.
 6. Theprocessing method for multiple depth information according to claim 5,wherein the first images are arranged in a vertical direction and thesecond images are arranged in a vertical direction.
 7. The processingmethod for multiple depth information according to claim 5, wherein thefirst images are arranged in a horizontal direction and the secondimages are arranged in a horizontal direction.
 8. The processing methodfor multiple depth information according to claim 1, wherein each of thefirst images and each of the second images are calculated by the depthgenerating unit in a time-division multiplexing manner to obtain each ofthe depth information.
 9. The processing method for multiple depthinformation according to claim 1, wherein a quantity of the first imagesis more than three and a quantity of the second images is more thanthree.
 10. The processing method for multiple depth informationaccording to claim 1, wherein a quantity of the first images is the sameas that of the second images.
 11. A processing system for multiple depthinformation, comprising: an image capturing module used to obtain aplurality of first images and a plurality of second images; and a depthgenerating unit used to receive the first images and the second imagesand calculate the first images and the second images to obtain aplurality of depth information corresponding to the first images and thetwo images.
 12. The processing system for multiple depth informationaccording to claim 11, wherein the image capturing module comprises: aplurality of image capturing units used to simultaneously capture aplurality of original images; and a plurality of conversion units usedto obtain the first images and the second images according to theoriginal images.
 13. The processing system for multiple depthinformation according to claim 12, wherein a quantity of the firstimages is two, a quantity of the second images is two and a quantity ofimaged captured by the image capturing units is two or three.
 14. Theprocessing system for multiple depth information according to claim 11,wherein the first images have different resolutions and the secondimages have different resolutions.
 15. The processing system formultiple depth information according to claim 11, further comprising: acombination unit used to combine the first images and the second images;wherein a plurality of depth information corresponding to the firstimages and the second images are obtained by the depth generating unitaccording to the composite image.
 16. The processing system for multipledepth information according to claim 15, wherein the combination unitarranges the first images in a vertical direction and arranges thesecond images in a vertical direction.
 17. The processing system formultiple depth information according to claim 15, wherein thecombination unit arranges the first images in a horizontal direction andarranges the second images in a horizontal direction.
 18. The processingsystem for multiple depth information according to claim 11, wherein thedepth generating unit calculates each of the first images and each ofthe second images to obtain each of the depth information in atime-division multiplexing manner.
 19. The processing system formultiple depth information according to claim 11, wherein a quantity ofthe first images is more than three and a quantity of the second imagesis also more than three.
 20. The processing system for multiple depthinformation according to claim 11, wherein a quantity of the firstimages is the same as that of the second images.